Device ancillary activity

ABSTRACT

Described herein is a system and method for performing ancillary activity. A device activity being performed by a user device of a user is detected. Attribute data associated with a plurality of attribute sources is accessed. A user preference indicating a preference for performing on a secondary user device a complementary activity corresponding to the device activity is inferred. Based on the inferred user preference, the secondary user device is identified according to a device status of the secondary user device, the device status indicating a device capability to perform the complementary activity. The complementary activity to be performed on the secondary user device is generated by analyzing at least one of the device activity, a device functionality of the secondary user device, and the user preference. Instructions to perform the complementary activity are transmitted to the secondary user device.

RELATED APPLICATIONS

This application is a continuation of and claims the benefit of priorityto U.S. patent application Ser. No. 15/431,799, filed on Feb. 14, 2017,which is a continuation of and claims the benefit of priority to U.S.patent application Ser. No. 14/498,326, filed on Sep. 26, 2014, whichclaims the benefit of priority of U.S. Provisional Application No.61/970,263, filed Mar. 25, 2014, each of which is claimed hereby, andeach of which are incorporated by reference herein in its entirety.

TECHNICAL FIELD

Embodiments of the present disclosure relate to data mesh-based wearabledevice ancillary activity.

BACKGROUND

In recent years, mobile devices, wearable devices, smart devices, andthe like have pervaded nearly every aspect of modern life. Such devicesare increasingly incorporating sensors to monitor everything from themoisture level of houseplants to a dribbling of a basketball.Network-connected devices like these are capable of providing a nearreal-time and constant data feed. These trends have provided a vastamount of rich, constantly updated data.

BRIEF DESCRIPTION OF THE DRAWINGS

Various ones of the appended drawings merely illustrate exampleembodiments of the present disclosure and should not be considered aslimiting its scope.

FIG. 1 is a block diagram illustrating a networked system, according tosome example embodiments.

FIG. 2A is a block diagram illustrating an example embodiment of a datamesh system, according to some example embodiments.

FIG. 2B is a block diagram illustrating an example embodiment of anancillary activity system, according to some example embodiments.

FIG. 3 illustrates an example of generating ancillary activity for asecondary user device, according to some example embodiments.

FIG. 4 is a flow diagram illustrating an example method for generatingancillary activity for a secondary user device, according to someexample embodiments.

FIG. 5 is a flow diagram illustrating further operations for inferring auser preference from the attribute data, according to some exampleembodiments.

FIGS. 6 and 7 are flow diagrams illustrating further operations forfacilitating identifying a secondary user device, according to someexample embodiments.

FIGS. 8 and 9 are flow diagrams illustrating further operations forgenerating an ancillary activity for a secondary user device, accordingto some example embodiments.

FIG. 10 illustrates an example scene showing a presentation of ancillaryactivity to the user, according to some example embodiments.

FIGS. 11 and 12 depict example user interfaces that present ancillaryactivity, according to some example embodiments.

FIGS. 13A and 13B depict example configurations for communicativelycoupling attribute sources, according to some example embodiments.

FIG. 14 depicts various example attribute sources, according to someexample embodiments.

FIG. 15 depicts various components that provide attribute data,according to some example embodiments.

FIG. 16 is a block diagram of an example data structure for exampleattribute data associated with a user, according to some exampleembodiments.

FIG. 17 is a block diagram of an example data structure for exampleattribute data associated with a device, according to some exampleembodiments.

FIG. 18 is a block diagram illustrating an example of a softwarearchitecture that may be installed on a machine, according to someexample embodiments.

FIG. 19 illustrates a diagrammatic representation of a machine in theform of a computer system within which a set of instructions may beexecuted for causing the machine to perform any one or more of themethodologies discussed herein, according to an example embodiment.

The headings provided herein are merely for convenience and do notnecessarily affect the scope or meaning of the terms used.

DETAILED DESCRIPTION

The description that follows includes systems, methods, techniques,instruction sequences, and computing machine program products thatembody illustrative embodiments of the disclosure. In the followingdescription, for the purposes of explanation, numerous specific detailsare set forth in order to provide an understanding of variousembodiments of the inventive subject matter. It will be evident,however, to those skilled in the art, that embodiments of the inventivesubject matter may be practiced without these specific details. Ingeneral, well-known instruction instances, protocols, structures, andtechniques are not necessarily shown in detail.

Modern users of technology typically have a variety of devices toperform various tasks such as laptop computers, smart televisions, smartphones, wearable devices. Traditionally, a user of such devices operatesa single device to perform a task at one time. The discussion belowdescribes systems and methods that, in some embodiments, utilizemultiple devices to perform, in real-time, an ancillary, complementary,or supplemental activity associated with a particular activity or task.In some embodiments, the systems and methods distribute a portion aparticular activity or task across multiple devices. The ancillary orsupplemental activity associated with a particular device activity maybe determined based on a variety of factors including an inferred userpreference, device functionality, and the device activity. Thus, theancillary activity is dynamically determined and performed for the userin real-time to assist the user with the device activity.

In various example embodiments, a device activity being performed inreal-time by a user device of a user is detected. For example, the usermay be browsing a website on a mobile device or a laptop computer. Oncethe device activity is detected, attribute data associated with the userfrom a plurality of attribute sources is accessed. In various exampleembodiments, the attribute data is received or accessed from a broadrange of attribute sources such as, for example, from mobile devices,smart devices, smart homes, social network services, user profiles,browsing histories, purchase histories, and so forth.

A user preference indicating a preference or desire of the user forperforming on a secondary user device an ancillary, supplemental,complementary, or companion activity corresponding to the deviceactivity. For example, an analysis of the attribute data can indicatethe user would like to view supplemental content corresponding to thedevice activity. In this example, the ancillary activity comprisespresenting the supplemental content to the user on the secondary userdevice.

Based on the inferred user preference, the secondary user device can beidentified according to a device status of the secondary user device,according to some example embodiments. In various implementations, thedevice status indicates a device capability to perform the ancillaryactivity in real-time. For example, if the ancillary activity comprisespresenting content to the user, the secondary user device may beincapable of performing such a task if the secondary user device is notwithin a vicinity of the user (e.g., within a presentation distance ofthe user).

Once the secondary user device is identified, the ancillary activity, tobe performed in real-time on the secondary user device, is generated byanalyzing the device activity, a device functionality of the secondaryuser device, and the user preference. For example, if the deviceactivity comprises providing directions to the user, the ancillaryactivity may comprise providing content associated with providingdirections such as a current heading or distance to destination readout.In various implementations, the ancillary activity is caused to beperformed on the secondary user device or transmitted to the secondaryuser device with instruction to perform in real-time the ancillaryactivity.

With reference to FIG. 1, an example embodiment of a high-levelclient-server-based network architecture 100 is shown. A networkedsystem 102 provides server-side functionality via a network 104 (e.g.,the Internet or wide area network (WAN)) to a client device 110. A user(e.g., user 106) may interact with the networked system 102 using theclient device 110. FIG. 1 illustrates, for example, a web client 112(e.g., a browser, such as the Internet Explorer® browser developed byMicrosoft® Corporation of Redmond, Wash. State), client application(s)114, and a programmatic client 116 executing on the client device 110.The client device 110 can include the web client 112, the clientapplication(s) 114, and the programmatic client 116 alone, together, orin any suitable combination. Although FIG. 1 shows one client device110, multiple client devices can be included in the network architecture100.

The client device 110 can comprise a computing device that includes atleast a display and communication capabilities that provide access tothe networked system 102 via the network 104. The client device 110comprises, but is not limited to, a remote device, work station,computer, general purpose computer, Internet appliance, hand-helddevice, wireless device, portable device, wearable computer, cellular ormobile phone, personal digital assistant (PDA), smart phone, tablet,ultrabook, netbook, laptop, desktop, multi-processor system,microprocessor-based or programmable consumer electronic, game consoles,set-top box, network PC, mini-computer, and the like. In further exampleembodiments, the client device 110 comprises one or more of a touchscreen, accelerometer, gyroscope, biometric sensor, camera, microphone,global positioning system (GPS) device, and the like.

The client device 110 can communicate with the network 104 via a wiredor wireless connection. For example, one or more portions of the network104 can be an ad hoc network, an intranet, an extranet, a VirtualPrivate Network (VPN), a Local Area Network (LAN), a wireless LAN(WLAN), a Wide Area Network (WAN), a wireless WAN (WWAN), a MetropolitanArea Network (MAN), a portion of the Internet, a portion of the PublicSwitched Telephone Network (PSTN), a cellular telephone network, awireless network, a Wireless Fidelity (Wi-Fi®) network, a WorldwideInteroperability for Microwave Access (WiMax) network, another type ofnetwork, or a combination of two or more such networks.

The client device 110 can include one or more of the applications (alsoreferred to as “apps”) such as, but not limited to, web browsers, bookreader apps (operable to read e-books), media apps (operable to presentvarious media forms including audio and video), fitness apps, biometricmonitoring apps, messaging apps, electronic mail (email) apps,e-commerce site apps (also referred to as “marketplace apps”), and soon. The client application(s) 114 can include various componentsoperable to present information to the user and communicate withnetworked system 102. In some embodiments, if the e-commerce siteapplication is included in the client device 110, then this applicationcan be configured to locally provide the user interface and at leastsome of the functionalities with the application configured tocommunicate with the networked system 102, on an as needed basis, fordata or processing capabilities not locally available (e.g., access to adatabase of items available for sale, to authenticate a user, to verifya method of payment). Conversely, if the e-commerce site application isnot included in the client device 110, the client device 110 can use itsweb browser to access the e-commerce site (or a variant thereof) hostedon the networked system 102.

In various example embodiments, the users (e.g., the user 106) can be aperson, a machine, or other means of interacting with the client device110. In some example embodiments, the users may not be part of thenetwork architecture 100, but may interact with the network architecture100 via the client device 110 or another means. For instance, the userscan interact with client device 110 operable to receive inputinformation from (e.g., using touch screen input or alphanumeric input)and present information to (e.g., using graphical presentation on adevice display) the users. In this instance, the users may, for example,provide input information to the client device 110 to be communicated tothe networked system 102 via the network 104. The networked system 102,in response to the received input information, communicates informationto the client device 110 via the network 104 to be presented to theusers. In this way, the user may interact with the networked system 102using the client device 110.

An Application Program Interface (API) server 120 and a web server 122may be coupled to, and provide programmatic and web interfacesrespectively to, one or more application server(s) 140. In variousimplementations, the application server(s) 140 hosts one or morepublication system(s) 142, payment system(s) 144, and a data mesh system150, each of which comprises one or more modules or applications andeach of which are embodied as hardware, software, firmware, or anycombination thereof. The application server(s) 140 are, in turn, shownto be coupled to one or more database server(s) 124 that facilitateaccess to one or more information storage repositories or database(s)126. In an example embodiment, the database(s) 126 are storage devicesthat store information to be posted (e.g., publications or listings) tothe publication system(s) 142. The database(s) 126 stores digital goodsinformation in accordance with some example embodiments.

Additionally, a third party application 132, executing on third partyserver(s) 130, is shown as having programmatic access to the networkedsystem 102 via the programmatic interface provided by the API server120. For example, the third party application 132, utilizing informationretrieved from the networked system 102, supports one or more featuresor functions on a website hosted by the third party. The third partywebsite, for example, provides one or more promotional, marketplace, orpayment functions that are supported by the relevant applications of thenetworked system 102.

The publication system(s) 142 provides a number of publication functionsand services to the users that access the networked system 102. Thepayment system(s) 144 likewise provide a number of functions to performor facilitate payments and transactions. While the publication system(s)142 and payment system(s) 144 are shown in FIG. 1 to both form part ofthe networked system 102, it will be appreciated that, in alternativeembodiments, each system 142 and 144 may form part of a payment servicethat is separate and distinct from the networked system 102. In someexample embodiments, the payment system(s) 144 may form part of thepublication system(s) 142.

The data mesh system 150 provides functionality to receive, retrieve, orstore a broad spectrum of data associated with the users, according tovarious embodiments. It will be noted that the collective, aggregatedattribute data is sometimes referred to as a “data mesh.” The data meshsystem 150 stores, for example, received data in storage devices such asthe database(s) 126. In an example embodiment, the data mesh system 150includes an ancillary activity system 152 that generates ancillaryactivities based on various triggers and information. In some exampleembodiments, the data mesh system 150 communicates with the clientdevice 110, the third party server(s) 130, the publication system(s) 142(e.g., retrieving listings), and the payment system(s) 144 (e.g.,purchasing a listing). In an alternative example embodiment, the datamesh system 150 can be a part of the publication system(s) 142.

Further, while the client-server-based network architecture 100 shown inFIG. 1 employs a client-server architecture, the present inventivesubject matter is, of course, not limited to such an architecture, andcan equally well find application in a distributed, or peer-to-peer,architecture system, for example. The various systems of theapplications server(s) 140 (e.g., the publication system(s) 142 and thepayment system(s) 144) can also be implemented as standalone softwareprograms, which do not necessarily have networking capabilities.

The web client 112 can access the various systems of the networkedsystem 102 (e.g., the publication system(s) 142) via the web interfacesupported by the web server 122. Similarly, the programmatic client 116and client application(s) 114 can access the various services andfunctions provided by the networked system 102 via the programmaticinterface provided by the API server 120. The programmatic client 116can, for example, be a seller application (e.g., the Turbo Listerapplication developed by eBay® Inc., of San Jose, Calif.) to enablesellers to author and manage listings on the networked system 102 in anoff-line manner, and to perform batch-mode communications between theprogrammatic client 116 and the networked system 102.

FIG. 2A is a block diagram of the data mesh system 150, which canprovide functionality to receive, retrieve, or access attribute datafrom attribute sources, analyze the attribute data, manage the attributedata, and so forth. In an example embodiment, the data mesh system 150can include a presentation module 210, a communication module 215, anattribute module 220, a characteristic module 225, a management module230, and the ancillary activity system 152. FIG. 2B is a block diagramof the ancillary activity system 152, which can provide functionality togenerate ancillary activities based on various triggers and information.The ancillary activity system 152 includes an activity module 250, apreference module 255, and a device module 260, according to an exampleembodiment. All, or some, of the modules 210-260 of FIGS. 2A and 2B, cancommunicate with each other, for example, via a network coupling, sharedmemory, and the like. It will be appreciated that each module of modules210-260 can be implemented as a single module, combined into othermodules, or further subdivided into multiple modules. It will further beappreciated that in some embodiments the modules or functionality of theancillary activity system 152 is implemented in the data mesh system 150and the modules or functionality of the data mesh system 150 isimplemented in the ancillary activity system 152. Other modules notpertinent to example embodiments can also be included, but are notshown.

Referring to FIG. 2A, the presentation module 210 provides variouspresentation and user interface functionality operable to interactivelypresent and receive information from users. For example, thepresentation module 210 causes presentation of various notifications oruser interfaces that provide the user an option to make a purchaseassociated with the identified items. The presentation module 210presents or cause presentation of information using a variety of meansincluding visually displaying information and using other device outputs(e.g., acoustic, haptic). Interactively presenting is intended toinclude the exchange of information between a device and a user. Theuser can provide input to interact with the user interface in a varietyof ways including alphanumeric input, cursor input, tactile input, orother input (e.g., one or more touch screen, camera, tactile sensors,light sensors, infrared sensors, biometric sensors, microphone,gyroscope, accelerometer, or other sensors). It will be appreciated thatthe presentation module 210 provides many other user interfaces tofacilitate functionality described herein. Further, it will beappreciated that “presenting” as used herein is intended to includecommunicating information to another device with functionality operableto perform presentation using the communicated information.

The communication module 215 provides various communicationsfunctionality and web services. For example, the communication module215 provides network communication such as communicating with thenetworked system 102, the client device 110, and the third partyserver(s) 130. In various example embodiments, the network communicationoperates over wired or wireless modalities. Web services are intended toinclude retrieving information from the third party server(s) 130, thedatabase(s) 126, and the application server(s) 140. Informationretrieved by the communication module 215 comprises data associated withthe user (e.g., user profile information from an online account, socialnetwork service data associated with the user), data associated with oneor more items listed on an e-commerce website (e.g., images of the item,reviews of the item, item price), other data to facilitate thefunctionality described herein, and so on.

The attribute module 220 can receive, access, or retrieve a wide varietyof attribute data from many different attribute sources. For example,the attribute module 220 receives, retrieves, or accesses the attributedata from user devices or machines (e.g., the client device 110), socialnetwork services, the third party server(s) 130, the publicationsystem(s) 142, the payment system(s) 144, other applications servers, orother attribute sources. The attribute data, as used herein, is intendedto include raw data such as sensor data, profile data, social networkcontent, and so on.

In some example embodiments, the attribute module 220 extracts theattribute data from various sources. For instance, a payment history logof the user can include a tremendous amount of extraneous data. Theattribute module 220 can extract purchase information such as itempurchased, time, purchase price, seller, location, brand, and so forthfrom the payment history log of the user.

In further example embodiments, the attribute module 220 performsvarious functions to prepare or condition the attribute data foranalysis. For instance, the attribute module 220 standardizes theattribute data to facilitate analysis of the attribute data (e.g.,determine a normal form for the data to allow for comparison and othermathematical analysis). The attribute module 220 performs many otherfunctions to prepare the attribute data for analysis.

In various example embodiments, the attribute module 220 stores theattribute data in association with the user for subsequent analysis. Forexample, the attribute module 220 stores the attribute data in thedatabase(s) 126. The attribute data can be stored in conjunction with auser identifier such that the attribute module 220 subsequently uses theuser identifier to access the attribute data corresponding to aparticular user. The attribute module 220 accesses the stored attributedata using other schemes. For instance, the attribute module 220accesses a portion of the attribute data associated with a time, anitem, a user, a type of user, a particular attribute source, and soforth. In this way, the attribute module 220 accesses a portion ofattribute data according to various parameters from among a largequantity of the attribute data to access, identify, or find pertinentand relevant data.

The characteristic module 225 infers a user characteristic or multipleuser characteristics corresponding to the user based on an analysis ofat least a portion of the attribute data. Many schemes and techniquescan be employed to infer the characteristic from the attribute data. Forexample, a particular user characteristic can be a work location of theuser. The attribute data can include a plurality of locations (e.g., asdetermined by a GPS component of a user device used by the user) thatinclude time stamps. The work location of the user can be inferred basedon the consistency and timing of the locations included in the attributedata (e.g., during normal working hours, the user is typically at aparticular office building). Many different portions of attribute dataand combinations of portions of attribute data can be analyzed to infera wide variety of characteristics.

In various example embodiments, characteristics (e.g., the usercharacteristics), as used herein, are intended to include traits,qualities, actions, activities, attitudes, habits, behaviors, and thelike pertaining to a person or people. Inasmuch as the attribute datamay not necessarily pertain to a person (e.g., raw data such ascoordinates of a particular location), a characteristic (e.g., currentlocation of the user, disliking spicy food, having young children, beinga Star Trek fanatic) can be distinct from the attribute data.

The management module 230 provides management functions associated withthe attribute data. For example, the management module 230 provides theuser with functionality to edit, modify, update, or otherwise controlthe attribute data. For instance, the user removes undesired attributedata via the functionality provided by the management module 230. In afurther instance, the user specifies permissions for portions of theattribute data using the functionality provided by the management module230. The permissions allow or prohibit certain access or uses for theattribute data (e.g., the permission prohibits access to the attributedata by third parties). Various levels of access and abilities can begranted. In some example embodiments, the permissions persist for aperiod of time, and after expiration of the time period, the permissionsare revoked.

In further example embodiments, the management module 230 requestsconsent from the user to access portions of the attribute data or torequest permission for certain uses of the attribute data. For example,the management module 230 requests consent from the user to allow thirdparties to access portions of the attribute data. The management module230 requests a variety of other consents associated with various actionscorresponding to the attribute data.

In still further example embodiments, the management module 230 providesfunctionality to allow third parties to access the attribute data oruser characteristics. For example, the management module 230 provides aset of APIs that can be invoked by third parties to access the attributedata or user characteristics. As discussed above, in some exampleembodiments, permission or consent of the user is determined prior toproviding access to the attribute data.

Referring now to FIG. 2B, the activity module 250 in the ancillaryactivity system 152 provides functionality to generate the ancillary,supplemental, complementary, or companion activity corresponding to thedevice activity. For example, the user can be browsing a website and theactivity module 250 generates the ancillary activity comprisingproviding the user interactive content such as an option to share, like,post (e.g., tweet), and so forth the website. The activity module 250generates many other ancillary activities corresponding to the deviceactivity based on various triggers and information.

The preference module 255 provides functionality to infer a userpreference, from the attribute data, indicating a preference of the userassociated with performing on the secondary user device the ancillaryactivity corresponding to the device activity. For example, thepreference module 255 infers that the user desires or prefers aparticular type of content associated with a particular device activity.The preference module 255 also identifies other users that are similarto the user and infer the user preference based on the identifiedsimilar users. The preference module 255 employs a variety of schemesand techniques using a wide range of data to infer the user preference.

The device module 260 provides functionality to identify the secondaryuser device according to a device status of the secondary user device.In various implementations, the device status indicates a devicecapability to perform the ancillary activity in real-time. The devicemodule 260 retrieves, derives, determines, or otherwise obtains avariety of information associated with the user device and the secondaryuser device to facilitate the functionality herein. For example, thedevice module 260 determines available functionality of the secondaryuser device.

FIG. 3 illustrates a diagram 300 showing an example of generatingancillary activity for a secondary user device, according to someexample embodiments. User 310 may be using wearable computing devicessuch as wearable device 320 (e.g., Google Glass®) or wearable device 330(e.g., a smart watch). In this example, the wearable device 330 iscommunicatively to user device 350 via wireless signals such as signal340. In various implementations, the user device 350 is communicativelycoupled, via coupling 360, to the network 104, which is in turncommunicatively coupled to the networked system 102 including the datamesh system 150 (discussed above in connection with FIG. 2A) and theancillary activity system 152 (discussed above in connection with FIG.2B).

The user 310 may be operating or using the user device 350. The terms“operating,” “using,” “in service,” or “in use” as used herein areintended to include a particular user physically interacting with aparticular device, being capable of operating the particular devicewithin a short time period such as dormant device or a device in standby(e.g., a particular user carrying a mobile device on their personwithout present physically interaction is included in the terms“operating,” “using,” or “in use”), or otherwise utilizing theparticular device (e.g., a smart refrigerator, that is not within avicinity of the user, configured to track inventory levels and provideinventory data).

In the example diagram 300, the user 310 is carrying the user device 350that is communicatively coupled to the ancillary activity system 152.The activity module 250 detects the device activity of the user device350. For instance, the user 310 can be browsing a webpage, receivingdirections to a particular location, monitoring fitness activity such asa number of steps taken, and so forth using the user device 350.

Once the activity module 250 detects the device activity, the preferencemodule 255 infers the user preference from the attribute data accessedby the attribute module 220. The user preference indicates a preferenceof the user or desire of the user for performing, on a particular userdevice, a particular ancillary activity corresponding to the deviceactivity. For example, the preference module 255 infers that the user310 would like an option, presented on a particular wearable device, tofavorite, share, post (e.g., tweet) associated with a webpage the user310 is browsing on the device 350.

Based on the inferred user preference, the device module 260 identifiesthe secondary user device according to the device status of thesecondary user device. The device status indicates that the device iscapable of performing the ancillary activity. For example, the devicemodule 260 identifies the wearable device 330 as the secondary userdevice based on the device status corresponding to the wearable device330. In this example, the device module 260 determines the device statusof the wearable device 330 as being in use by the user 310 (e.g., theuser 310 wearing the wearable device 330). Thus, the wearable device 330is operable to perform the ancillary activity since providing the optionfor the user 310 to select can be effectuated by the wearable device 330as the user 310 is within an operating distance of the wearable device330.

After the device module 260 identifies the secondary user device, theactivity module 250 generates the ancillary activity to be performed inreal-time on the secondary user device. The activity module 250generates the ancillary activity by analyzing the device activity, adevice functionality of the secondary user device, the user preference,and other data. The device functionality indicates options available forinput and output on the secondary user device. For example, theidentified secondary user device is the wearable device 330 that has asmall screen area to display user interfaces or has reduced options foroutput (e.g., no speakers). In a specific instance, the activity module250 generates the ancillary activity to include abridged activitycontent according to the device functionality (e.g., small displaysize). After the activity module 250 generates the ancillary activity,the activity module 250 transmits or otherwise communicates to thesecondary user device to perform the ancillary activity in real-time.

FIG. 4 is a flow diagram illustrating an example method 400 forgenerating ancillary activity for the secondary user device, accordingto some example embodiments. At operation 410, the activity module 250detects the device activity being performed in real-time by the userdevice of a user. The term “real-time data,” as used herein, is intendedto include data associated with an event currently happening. Forexample, the device activity being performed in real-time includes aparticular device activity detected at the activity module 250 after adelay interval (e.g., due to transmission delay or other delays such asbeing temporarily stored at an intermediate device) between the instantof the particular device activity occurring and the activity module 250detecting the particular device activity. Thus, in some instances, thedevice activity being performed in real-time is intended to includeactivities that have occurred a short time in the past. This discussionof real-time applies equally throughout the specification in relation toother uses of the term “real-time.”

In various embodiments, the device activity includes a wide variety ofactivities such as browsing a webpage, monitoring fitness activity(e.g., steps the user has taken), heart rate monitoring, inventory levelmonitoring (e.g., a smart refrigerator that monitors inventory), and soon. In some implementations, the activity module 250 detects the deviceactivity being performed by monitoring devices of the user. Forinstance, a smart appliance of the user provides a continuous orperiodic stream of data indicating various device activities.

At operation 420, the attribute module 220 accesses the attribute dataassociated with the user from the plurality of attribute sources. Invarious example embodiments, at least a portion of the attribute dataincludes real-time data or near real-time data. For example, thereal-time data includes user input data or sensor data communicated tothe attribute module 220 after a delay interval (e.g., due totransmission delay or other delays such as being temporarily stored atan intermediate device) between capturing the data and the attributemodule 220 receiving the data.

As will be discussed in connection with FIGS. 14 and 15, the attributedata is received from a broad spectrum of attribute sources (e.g.,devices, sensors, servers, databases, and other sources). Additionally,the attribute module 220 receives or accesses the attribute data viamany pathways resulting from an assortment of configurations of theattribute sources as further discussed in connection with FIGS. 13A and13B. In an example embodiment, the attribute module 220 receives theattribute data directly from the attribute sources. In other exampleembodiments, the attribute module 220 receives the attribute data from acentral device that receives attribute data from a plurality of userdevices. In still other example embodiments, various user devices arecommunicatively coupled in a decentralized device-to-device mesh, andthe attribute module 220 receives the attribute data corresponding to aparticular device in the mesh from any of the devices in the mesh. Theattribute module 220 receives the attribute data from the attributesources in many other configurations including various suitablecombinations of configurations.

In various example embodiments, the attribute module 220 stores theattribute data in association with the user (e.g., indexed based on auser identifier) for subsequent analysis. The attribute module 220stores the attribute data in a storage device such as the database(s)126, for example. The attribute module 220 accesses the stored attributedata using a variety of search or find schemes. For instance, theattribute data associated with a particular user is accessed using auser identifier that corresponds to the particular user. It will benoted that the collective, aggregated attribute data is sometimesreferred to as a “data mesh.”

At operation 430, the preference module 255 infers the user preferenceor desired user setting, from the attribute data, indicating apreference of the user for performing on the secondary user device theancillary activity corresponding to the device activity. For example,the attribute data includes engagement data of the user (e.g.,particular websites the user visits, taps, clicks, or other interactionswith various notification) indicative of the types of information theuser is interested in. In a specific example, the preference module 255infers the user preference for receiving notifications associated with aparticular sporting event based on the engagement data.

In this specific example, the activity module 250 detects from the userdevice (e.g., a smart television) that the device activity includeswatching a sporting event. In continuing with this example, thepreference module 255 infers the user preference to indicate the userhas an affinity for notification associated with watching a sportingevent on a particular secondary device based on past notifications theuser showed interest in that are included in the attribute data.

Referring now to FIG. 5, a flow diagram is shown illustrating furtheroperations for inferring a user preference from the attribute data,according to some example embodiments. As discussed above, subsequent tothe operation 420, at the operation 430, the preference module 255infers the user preference from the attribute data.

At operation 510, the characteristic module 225 infers or directlymeasures user characteristics pertaining to the user from the attributedata. In some example embodiments, the characteristic module 225 storesthe inferred user characteristics for subsequent analysis, for example,in a storage device such as database(s) 126. The characteristic module225 infers a vast spectrum of the user characteristics from theattribute data. A few specific examples of user characteristics includedemographic data (e.g., age, gender, marital status, number ofchildren), user preferences (e.g., being a morning person, favoritelocations, enjoying spicy food), idiosyncrasy (e.g., being forgetful,such as draining the battery on a mobile device; or being impatient,such as a line breaker that will leave a store if the line is too long),qualities (e.g., being athletic, being tall, having a large vocabulary),personality traits (e.g., being a risk taker), actions, activities(e.g., working for a non-profit), attitudes, habits (e.g., being acoffee drinker), behaviors, beliefs, biases, demeanor, and physicalcharacteristics of the user (e.g., height, weight, garment sizes, eyecolor, hair color). The specificity of the characteristics ranges fromvery narrow (e.g., drinks a particular brand of soda) to very broad(e.g., being generally philanthropic). To illustrate inferring the usercharacteristic from the attribute data in an example, the attribute dataincludes user location data that indicates frequent visits to a localschool, local soccer fields, and the like. In this example, thecharacteristic module 225 infers that the user has children based on thetypes of locations the user may be frequently visiting.

In some instances, the characteristic module 225 performs varyingdegrees of inferential analysis of the attribute data to derive the usercharacteristics. For example, the characteristic module 225 infers theuser's wake-up time based on user device activity or other activity(e.g., connected alarm clock settings, logins to accounts, and variousother user activities that indicate a wake-up time). In this example,the characteristic module 225 infers a particular user characteristicthat can be of a larger inferential jump such as the user being amorning person or a person that likes to sleep in. The degree ofinferential jump can be configurable. In some example embodiments, thecharacteristic module 225 employs various techniques to minimize orotherwise control incorrect inferences (e.g., machine-learning, otherlearning algorithms).

In further example embodiments, the characteristic module 225 learns,adapts, or evolves as more of the attribute data is received (e.g., viamachine learning techniques or other learning algorithms). For example,the attribute data includes location data of the user. Thecharacteristic module 225 infers a favorite location of the user basedon a pattern (e.g., frequently visited locations) in the location data.However, the characteristic module 225 subsequently receives employmentdata of the user that indicates a current employer including an employerlocation. The characteristic module 225 learns, updates, or otherwiseadapts to account for the new attribute data. Thus, in this example, thecharacteristic module 225 may not infer a favorite location of the userif the location is a work location of the user. In some instance, theuser may provide input directly (e.g., via a user interface configuredto receive inferential guidance from the user) to facilitate thecharacteristic module 225 in inferring characteristics from theattribute data (e.g., user input indicating that a particular inferredcharacteristic is incorrect or providing input to be used as a basis forfuture inferences).

In other instances, the characteristic module 225 performs very littleor no analysis to derive the user characteristic from the attributedata. For example, the attribute data includes an alarm time settingfrom a connected alarm clock (e.g., a smart phone with an alarm clockapp). The alarm time setting can directly indicate a wake-up time. Sincethe attribute data directly relates to a particular user characteristic,the characteristic module 225 need not perform analysis to derive theuser characteristic.

In some example embodiments, the user characteristic comprisespredefined characteristics or dynamically determined characteristics.For instance, a particular set of characteristics is predefined (e.g.,work location, home location, marital status, socio-economic level). Inthis instance, the characteristic module 225 determines that particularpredefined characteristics are associated with the user based on ananalysis of the attribute data. In other instances, the characteristicmodule 225 dynamically determines characteristics based on the attributedata. For example, the attribute data indicates that the user owns aparticular exotic pet. Although there may not be a predefinedcharacteristic associated with the particular exotic pet, thecharacteristic module 225 determines the user characteristic of owningan exotic pet from the attribute data.

At operation 520, the preference module 255 identifies similar usersthat are similar to the user based on the inferred user characteristicsand respective user characteristics of a plurality of other users. Thepreference module 255 identifies similar users that are similar to theuser based on a variety of factors. In some example embodiments, thepreference module 255 accesses the attribute data or stored usercharacteristics corresponding to the plurality of other users. Forexample, the preference module 255 identities the similar users fromamong the plurality of other users that are similar to the user based onthe inferred user characteristics of the user and respective usercharacteristics of the plurality of other users. The preference module255 correlates, matches, or otherwise compares the inferred usercharacteristics with respective user characteristics of the plurality ofother users to identify the similar users. In various exampleembodiments, the preference module 255 identifies the similar usersbased on same or similar demographic data (e.g., same or similar age,marital status, gender, geographic location, etc.), same or similar usercharacteristics (e.g., same or similar brand purchases), same or similarattribute data, and so on.

At operation 530, the preference module 255 infers the user preferenceor desired user setting based on the identified similar users. Forexample, the preference module 255 analyzes the user characteristics ofthe identified similar users to determine the user preference. In aspecific example, if the user characteristics of the identified similarusers indicate a preference for viewing particular content on aparticular secondary user device, the preference module 255 infers thatthe user also has the same or a similar preference for viewing theparticular content on the particular secondary user device. In this way,the preference module 255 infers the user preference based on theidentified similar users.

Referring back to FIG. 4, at operation 440, the device module 260identifies the secondary user device according to the device status ofthe secondary user device. The device status indicates a devicecapability to perform the ancillary activity in real-time. The devicecapability includes various metrics and features associated with thesecondary user device. In various implementations, the device module 260identifies the secondary user device based on the device status thatincludes various combinations of factors discussed below. For example,the device status can be based on the device capability including adistance of the secondary user device to the user as further discussedin connection with FIG. 6, below. For instance, the secondary userdevice is capable of presenting information or notification to the userif the secondary user device is within a vicinity of the user.

In another example, the device status is based on a particularfunctionality of the device. The device module 260 queries the secondaryuser device to determine the functionality of the secondary user device.In other instance, the device module 260 determines the functionality ofthe secondary user device via a lookup of functionality for the same orsimilar device as the secondary user device. For instance, if thesecondary user device is a particular type of wearable device, thedevice module 260 can lookup available functions for the particular typeof wearable device from the third party servers(s) 130. For instance, ifthe secondary user device has an audio output, then the device module260 identifies the secondary user device as capable of performing aparticular ancillary activity that includes audio output (e.g., voicedirections for hands free navigation). Thus, in some cases, the devicemodule 260 determines the device status in conjunction with a particularancillary activity.

In still another example, the device status is based on whether thesecondary user device is active as further discussed in connection withFIG. 7, below. For example, if the secondary user device is notcurrently being used (e.g., as determined via a sensor on the devicesuch as an accelerometer that indicates the device is perfectly still),then the device module 260 determines that the device is not active anddoes not identify the inactive secondary user device as being capable ofperforming the ancillary activity.

Referring now to FIG. 6, a flow diagram illustrates further operationsfor identifying a secondary user device, according to some exampleembodiments. Specifically, FIG. 6 is directed to the device module 260identifying the secondary user device by determining that the devicestatus of the secondary user device includes the secondary user beingwithin a distance of the user. Subsequent to the operation 430, thedevice module 260 receives sensor data from the secondary user device atoperation 610. The sensor data includes, for example, data received fromany of the sensors discussed in connection with FIG. 15, below. In aspecific example, the sensor data includes location data as determinedby a GPS component of the secondary user device. The sensor datarepresents a real-time physical environment of the secondary userdevice. As discussed below, the device module 260 determines the devicestatus of the secondary user device based on the sensor data receivedfrom the secondary user device.

At operation 620, the characteristic module 225 or the device module 260infers the current user location based on location data received fromthe user device. For example, the attribute data includes real-timedata, from the user device, corresponding to the location of the user(e.g., location data as determined by a GPS component of a mobile devicecorresponding to the user).

At operation 630, the device module 260 extracts a current devicelocation from the sensor data received from the secondary user device.For instance, the secondary user device can be equipped with a GPScomponent that provides location data. In other instances, the devicemodule 260 extracts the current device location based on Wi-Fi®triangulation, NFC beacon detection, or other location services.

At operation 640, the device module 260 compares the current userlocation and the current device location to determine that the secondaryuser device is within a distance (e.g., an operating distance) of thecurrent user location. The distance may be a short distance such as areasonable distance to allow the user to physically operate thesecondary user device (e.g., an arm's length).

Although FIG. 6 is directed to determining that the secondary userdevice is within a distance of the user location based on an inferredlocation of the user and the secondary user device, the device module260 employs other schemes to determine whether the user is within anoperating distance of the secondary user device. For instance, if thesecondary user device is equipped with biometric identification sensorsand the device module 260 receives biometric sensor data from thesecondary user device indicating the identity of the user, the devicemodule 260 can, in that situation, infer that the user is within anoperating distance of the secondary user device.

Referring now to FIG. 7, a flow diagram illustrates further operationsfor identifying the secondary user device, according to some exampleembodiments. Subsequent to the operation 430, the device module 260receives sensor data from the secondary user device at operation 710,similar to the operation 610 described above. The sensor data representsa real-time physical environment of the secondary user device. Forinstance, the sensor data includes thermal data (e.g., data thatindicates a current temperature), motion data (e.g., as determined by anaccelerometer component), position data (e.g., as determined by a GPScomponent), biometric data (e.g., heart rate data or fingerprintidentifications), communication data (e.g., NFC beacon detections orBluetooth® device detections), and other sensor data (see FIG. 15 belowfor additional sensors and data).

At operation 720, the device module 260 calculates an active metric fromthe sensor data. In various implementations, the active metric indicatesthe secondary user device is in use by the user. In someimplementations, the active metric comprises a probability or likelihoodthat the secondary user device is active. In these implementations, ahigher value for the active metric is associated with a higherprobability that the secondary user device is active. In a specificinstance, if the secondary user device is a wearable device such as asmart watch, the sensor data indicates that a particular user is wearingthe wearable device based on a heart rate sensor (when the user is notwearing the wearable device the heart rate sensor indicates no heartrate).

Similarly, if the secondary user device is a smart phone that providesambient temperature data, the device module 260 calculates the activemetric based on the temperature data. For example, if the temperaturedata is fluctuating or is above an expected ambient temperature, thisindicates that the user is carrying the smart phone in their pocket.Conversely if the temperature is near expected ambient temperature andthere is little fluctuation in the temperature, the device module 260calculates a low probability, corresponding to a lower active metric,indicating that the particular device is not active or in use by theuser.

At operation 730, the device module 260 determines that the secondaryuser device is active based on the active metric. For example if thecalculated active metric exceeds a threshold value, the device module260 determines that the secondary user device is active. The thresholdvalue can be predetermined or dynamically determined by the devicemodule 260. For example, the device module 260 employs variousstatistical models based on historical values of the active metric todetermine whether the current active metric is anomalous. In a simplenon-limiting example, the device module 260 determines the thresholdvalue to be an average of historical values for the active metric, andthe device module 260 determines that the secondary user device isactive if the active metric exceeds the average of the historical valuesof the active metric. The device module 260 employs many other schemesand techniques to determine that the device is active based on theactive metric.

At operation 740, the device module 260 identifies the secondary userdevice as being capable of performing the ancillary activity inreal-time based on the secondary user device being active. That is tosay, the device module 260 identifies a particular secondary user devicebased on the particular secondary user device being active. Thereasoning being that if the secondary user device is inactive, thesecondary user device is not capable of presenting information to orreceiving input from the user and therefore is not capable of performingthe ancillary activity, in some cases.

Referring again to FIG. 4, at operation 450, the activity module 250generates the ancillary activity, to be performed in real-time on thesecondary user device, by analyzing the device activity, a devicefunctionality of the secondary user device, the user preference, andother factors and data. The ancillary activity can include a wide rangeof tasks, content, and functionality. For example, the ancillaryactivity comprises a notification that includes notification contentassociated with the device activity. For instance, if the deviceactivity is viewing a sporting event on a smart TV, the ancillaryactivity comprises a notification including notification contentassociated with the sporting event (e.g., information about the players,the current score, sporting statistics).

In some implementations, the ancillary activity includes a portion ofthe device activity. For example, if the device activity comprisesbrowsing a particular website, the activity module 250 generates theancillary activity to include content that is a portion of the website.Thus, in these implementations, the activity module 250 is distributinga portion of the device activity to the secondary user device.

FIG. 8 is a flow diagram illustrating further operations for generatingthe ancillary activity of the secondary user device, according to someexample embodiments. Subsequent to the operation 440, at the operation450, the activity module 250 generates the ancillary activity. Theoperation 450 can further include operations 810, 820, and 830.

At operation 810, the device module 260 determines a display sizecorresponding to the secondary user device. For example, the devicemodule 260 queries the secondary user device directly to retrieve dataassociated with the display size. In another example, the device module260 queries a particular third party server (e.g., the third partyserver(s) 130) using an identifier associated with the secondary userdevice to determine the display size of the secondary user device. In aspecific example, the device module 260 determines a particular devicemodel for the secondary user device and performs a lookup of the displaysize for the particular device model.

At operation 820, the device module 260 determines the display sizecorresponding to the secondary user device is below a threshold size.For example, the display size may be too small for particular content ofthe ancillary activity. In this situation, the device module 260determines that the display size is below the threshold size.

At operation 830 the activity module 250 generates the ancillaryactivity to include abridged activity context according to the displaysize corresponding to the secondary user device. For instance, thedevice module 260 shortens or reduces content or functionality of theancillary activity to accommodate the display size of the secondary userdevice.

FIG. 9 is a flow diagram illustrating further operations for generatingthe ancillary activity of the secondary user device, according to someexample embodiments. Subsequent to the operation 440, at the operation450, the activity module 250 generates the ancillary activity. Theoperation 450 further includes operations 910 and 920.

At operation 910, the device module 260 compares the devicefunctionality of the secondary user device with a device functionalityof the user device to identify non-mutual functionality of the secondaryuser device that is not available on the user device. For example, theuser device comprises a mobile computer that the user is browsing awebsite on. The secondary user device can be a wearable device affixedto the user. In this example, the device module 260 identifies hapticoutput, for instance, of the secondary user device as being non-mutualfunctionality as the mobile computer may not have haptic outputfunctionality. In another example, a particular wearable device may notinclude a GPS component while a smart phone device of the user mayinclude a GPS component to determine current location. In this example,the device module 260 identifies use of the GPS component as non-mutualfunctionality.

At operation 920, the activity module 250 generates the ancillaryactivity to include an activity component that utilizes the non-mutualfunctionality of the secondary user device. In the examples discussedabove, if the device module 260 determines that the secondary userdevice includes a GPS component while the user device does not include aGPS component, the activity module 250 utilizes location data from thesecondary user device in generating the ancillary activity. In aspecific example, if the user is browsing a particular webpageassociated with a particular merchant or location, the activity module250 generates the ancillary activity to include mapping directions tothe particular location using the GPS component of the secondary userdevice to access the user's current location. Thus, the activity module250 receives the particular location from the user device andcommunicates, to the secondary user device, the location andinstructions to map directions to that location from the currentlocation.

Turning again to FIG. 4, at operation 460, the activity module 250transmits or otherwise communicates, to the secondary user device,instructions to perform in real-time the complementary activity. Forexample, the activity module 250 communicates instructions to thesecondary user device to present a user interface that facilitates oreffectuates the ancillary activity (e.g., a user interface includingfunctionality to carry out the ancillary activity). In a specificexample, if the ancillary activity comprises a notification includingnotification content, the activity module 250 transmits the notificationincluding the notification content to the secondary user device to bepresented to the user. Thus, in this example, the activity module 250caused presentation of the notification on the secondary user device.

Although the operations 440-460 are directed to performing a singleancillary activity on a single secondary user device, other embodimentsinclude identifying multiple secondary devices and generating multipleancillary activities to be performed in real-time on one or more of theidentified secondary user devices. For example, if the user is carryinga smart watch and wearing smart glasses, the generated ancillaryactivities can be distributed among the identified ancillary activities.For instance, haptic-based notifications can be directed to the user'ssmart watch while visual notification is directed to the user's smartglasses. In this instance, the activity module 250 distributes themultiple ancillary activities among the multiple secondary user devicesbased on respective device functionalities of the multiple secondaryuser devices. The activity module 250 employs other factors todistribute the multiple ancillary activities such as the userpreference.

To help illustrate the concepts described above, FIG. 10 illustratesnon-limiting examples of generating the ancillary activity for thesecondary user device, according to some example embodiment. FIG. 10includes a scene 1000 that depicts a living room attached to an openkitchen. The scene 1000 includes a smart television (TV) 1010, a mediaentertainment device 1020, a lamp 1030, a mobile computer 1040, awearable device 1050, a user 1060, a mobile device 1070, a smartrefrigerator 1080, and a kitchen display 1090. Each of the devices ofFIG. 10 can be attribute sources coupled to a network (e.g., the network104) and operable to communicate with the data mesh system 150. Invarious example embodiments, the user 1060 may be carrying or wearing asmart device such as the wearable device 1050 (e.g., a mobile device, awearable device, a NFC enable smart ring) on their person that providesreal-time data corresponding to the user 1060. For instance, the user1060 may be carrying a mobile device that provides real-time locationdata (e.g., as determined by a GPS component, beacon location detect, orother location services).

In an example embodiment, the user 1060 may be browsing a webpage usingthe mobile computer 1040. The mobile computer 1040 can be coupled to thenetwork 104 and the activity module 250 detects the device activity ofbrowsing a particular webpage. The preference module 255 infers a userpreference associated with the device activity of browsing a particularwebpage. For instance, the activity module 250 infers that the user hasa preference for viewing supplemental content associated with theparticular webpage the user is browsing or supplemental functionalityassociated with the particular webpage (e.g., liking, favoriting, orsharing the particular webpage).

Based on the inferred user preference, the device module 260 identifiesthe secondary user device such as the wearable device 1050 that iscapable of performing the ancillary activity. The device module 260identifies the wearable device 1050 based on the wearable device 1050being within an operating distance of the user 1060 (the reasoning beingthat if the wearable device 1050 is within a short distance of the user,then the wearable device 1050 is capable of providing information orreceiving input from the user). For example, the device module 260determines the user is wearing the wearable device 1050 based on varioussensor data received from the wearable device 1050 (e.g., biometric datathat indicates a particular user is wearing the device, accelerometerdata that indicates the device is in use, Bluetooth® device detectionsthat indicates the wearable device 1050 is within short rangecommunication range of another user device that the device module 260has inferred the user is using).

Once the device module 260 identifies the wearable device 1050, theactivity module 250 generates the ancillary activity based on the devicefunctionality of the identified secondary user device, the deviceactivity, the user preference, and other data. For example, the wearabledevice 1050 may not have an audio output; in that case, contentincluding an audio component can be altered or modified to accommodatethe functionality of the wearable device 1050. In continuing with theexample above, the activity module 250 generates the ancillary activityto include options to favorite, like, share (e.g., tweet) the particularwebpage the user is browsing. In some instances, the activity module 250generates the ancillary activity based on the device activity. Forexample, if the device activity of browsing a particular webpage alreadyincludes options to favorite or share the webpage, the activity module250 generates the ancillary activity to include functionality notalready available at the user device (the mobile computer 1040 in thiscase). After the activity module 250 generates the ancillary activity,the activity module 250 transmits or otherwise communicates to thewearable device 1050 to perform the ancillary activity. In this example,the activity module 250 transmits instructions to the wearable device1050 to present a user interface including the options to favorite,like, or share the particular webpage the user is browsing on the mobilecomputer 1040.

Example User Interfaces

FIGS. 11 and 12 depict example user interfaces for interactivelypresenting information to the user. Although FIGS. 11 and 12 depictspecific example user interfaces and user interface elements, these aremerely non-limiting examples and many other alternate user interfacesand user interface elements can be generated by the presentation module210 and presented to the user. It will be noted that alternatepresentations of the displays of FIGS. 11 and 12 can include additionalinformation, graphics, options, and so forth; other presentationsinclude less information, or provide abridged information for easy useby the user.

FIG. 11 depicts an example device 1100 (e.g., smart watch) displaying anexample user interface 1110 that includes ancillary or supplementalfunctionality. For example, the user interface 1110 includes userinterfaces elements 1120, 1130, and 1140 that provide the user withoptions to like (e.g., like a particular webpage or a particularlocation), share (e.g., tweet a link to a particular location or awebpage), or return to home (e.g., navigate a particular webpage on theuser device using the secondary user device) respectively. For example,activating the user interface element 1120 causes the secondary userdevice to “like” an item associated with the device activity (e.g., aparticular webpage) on a social network service the user is a member of.The user interface 1110 includes a wide variety of other functionality.

In some cases, the ancillary activity comprises a notification. In thesecases, the activity module 250 causes presentation of the notificationto the user. For instance, the activity module 250 communicates, to thedevice 1100, instructions to present the notification. In someinstances, the instructions include notification content, generated bythe activity module 250, such as a message (e.g., pertinent information)to be presented to the user. In example embodiments, the notificationcomprises a text message, such as Short Message Service (SMS) messages,Multimedia Messaging Service (MMS), Enhanced Messaging Service (EMS),and so forth. In other example embodiments, the notification comprises apush notification or another similar type of notification. In furtherexample embodiments, the notification comprises interactive userinterface elements.

FIG. 12 depicts an example device 1200 displaying an example userinterface 1210 that includes ancillary or supplemental content. Forexample, the user interface 1210 includes supplemental contentassociated with directions to a particular location. For instance, ifthe user is attempting to physically locate a particular place using asmart phone, the ancillary activity includes providing additional orsupplemental content on a wearable device the user may be wearing. Inthe example of FIG. 12, the user interface 1210 includes a headingdetermined via sensors included in the wearable device or from datareceived from the smart phone. The supplemental content can also includepertinent information such as a distance to the specified location. Inthis way, the wearable device performs the ancillary activity associatedwith the device activity being performed in real-time by the user deviceof the user.

FIGS. 13A and 13B depict example configurations for communicativelycoupling attribute sources, according to some example embodiments. Theexample embodiments described herein can access a vast and rich“Internet of Things” (IoT) dataset that is predominantly provided viaconnected, interconnected, or otherwise communicatively coupled machinesand devices that can include a multitude of sensors. In exampleembodiments, devices and machines that provide the attribute data, suchas the attribute sources, can be communicatively coupled in manydifferent configurations. For instance, each attribute source iscommunicatively coupled to the networked system 102 independently toprovide the networked system 102 access to the attribute datacorresponding to each of the communicatively coupled attribute sources.FIGS. 13A and 13B depict alternative example attribute sourceconfigurations. It will be appreciated that FIGS. 13A and 13B are merelynon-limiting examples of attribute source configurations and many otherconfigurations or suitable combinations of configurations can beemployed.

FIG. 13A depicts an example embodiment that includes attribute sources1310 communicatively coupled in a decentralized device-to-device mesh.In this example embodiment, the attribute data corresponding to aparticular device in the mesh may be received from any one or more ofthe devices in the mesh. For instance, the networked system 102 mayaccess the attribute data corresponding to attribute source E viaattribute source H or a combination of attribute sources H and I in FIG.13A. In an example embodiment, the attribute source H or I may aggregateand store the attribute data corresponding to attribute sources A-F inFIG. 13A. In some example embodiments, the networked system 102 mayaccess the attribute data associated with attribute source E bycommunicating with attribute source H or I in FIG. 13A.

FIG. 13B depicts another example embodiment that may include attributesources 1320 communicatively coupled to a central attribute source(e.g., attribute source H in FIG. 13B). The networked system 102 mayaccess the attribute data associated with attribute sources A-G via thecentral attribute source in FIG. 13B. In some embodiments, the centralattribute source may aggregate and store the attribute data received oraccessed from the attribute sources A-G and provide a centralized accesspoint for the attribute data associated with all, or some, of thecommunicatively coupled attribute sources A-G in FIG. 13B.

FIG. 14 depicts example sources 1400 including attribute sources 1410,according to some example embodiments. In various example embodiments,the attribute data may include data received, retrieved, or accessedfrom the attribute sources 1410. For example, the attribute sources 1410may provide data including everything from a moisture level of ahouseplant to a dribbling rhythm of a basketball. In some embodiments,the attribute data corresponding to the attribute sources 1410 may bereceived or accessed in real-time or near real-time. For instance, theattribute sources 1410 may communicate or otherwise provide access tothe attribute data as it becomes available. In example embodiments, theattribute sources 1410 may include user device sources 1420, user datasources 1430, transportation sources 1440, materials sources 1450, thirdparty sources 1460, home sources 1470, and a variety of other sources.As will be discussed in connection with FIG. 15, the attribute sources1410 may be associated with a wide variety of sensors, gauges,measurement components, and other components.

In an example embodiment, the attribute data may include datacorresponding to the user device sources 1420. The user device sources1420 may include such non-limiting examples as a personal computer (PC),a tablet computer, a laptop computer, a netbook, a set-top box (STB), apersonal digital assistant (PDA), an entertainment media system, acellular telephone, a smart phone, a mobile device, a wearable device(e.g., a smart watch), a smart home device (e.g., a smart appliance),and other smart devices. As will be discussed further in connection withFIG. 15, the attribute data corresponding to the user device sources1420 may include data associated with sensors, gauges, or othermeasurement components such as environmental sensor data (e.g., ambienttemperature data associated with an environment of the user), biometricsensor data (e.g., heart rate data of the user), detection data (e.g.,detection of a Near Field Communication (NFC) beacon), motion data(e.g., acceleration data), position data (e.g., location as determinedby a GPS of a mobile device), and so forth.

In further example embodiments, the attribute data corresponding to theuser device sources 1420 includes data such as device type, devicemodel, device name, a unique device identifier, and other deviceparameters. In some example embodiments, the device type data provides abasis for an inference associated with the attribute data. For instance,if the device type data indicates that the device is a mobile device ofthe user, location data corresponding to the mobile device may indicatethe location of the user. Similarly, if the device type is a mediaentertainment system, the attribute data corresponding to the mediaentertainment system may be associated with a home of the user.

The user data sources 1430 include, for example, calendars (e.g., usercalendar events such as birthdays, trips, exams), user profiles (e.g.,demographic information such as age, gender, income level), purchasehistories, browse histories (e.g., search terms), social media content(e.g., check-ins, posts, connections), or other user data (e.g.,bookmarked websites, preferences or settings for various applications,application usage data such as time spent using a particularapplication). The attribute data corresponding to the user data sources1430 is stored, for example, by the user device sources 1420 (e.g., amobile device that includes a mobile browser with browse history of theuser), application server(s) 140 (e.g., payment history of the userstored in payment system(s) 144, user profiles stored by an e-commercewebsite), or the third party server(s) 130 (e.g., social media datastored in a social networking service). For instance, the attribute datacorresponding to the user device sources 1420 includes device resourcedata. In some implementations, the device resource data includes filesstored on the devices (e.g., digital media or apps) or metadataassociated with the files (e.g., the number of time a particular songhas been played or usage time corresponding to a particular app).

As cars and other forms of transportation become increasingly equippedwith sensors and the ability to communicate, a vast amount of data maybe provided by the transportation sources 1440. For example, theattribute data corresponding to the transportation sources 1440 mayinclude acceleration data, velocity data, and other sensor data (e.g.,brake pad wear data, gear shifting data, miles driven). In this example,the attribute data corresponding to the transportation sources 1440 mayprovide indications of a user's driving patterns and styles (e.g., comesto a complete stop at a stop sign, speeds, or finicky use of thebrakes).

The materials sources 1450, such as clothing and structures, are alsoincreasingly gaining the ability to capture data. In various exampleembodiments, the attribute data may include data corresponding to thematerials sources 1450. For example, clothing may be embedded withsensors to detect motion. Data from these sensors may provideindications of whether the user is active or inactive. In anotherexample, clothing may be embedded with biometric sensors that mayprovide a continuous feed of biometric data corresponding to the user.The biometric data may provide indications of the user's health,athletic ability, and many other characteristics corresponding to theuser. Similarly, structures may be equipped with sensors to passively oractively monitor the surrounding environment (e.g., street cameras,traffic cameras, and other sensors).

In example embodiments, the attribute data may include data associatedwith the third party sources 1460. The third party sources 1460 may alsoprovide an abundance of data associated with the user. For instance, theattribute data may include data accessed from government websites orother public records that may provide criminal histories, civil citationhistories, credit histories, or other publicly available information.

In various embodiments, a smart home is a house, office, or otherenvironment of the user with one or more smart devices integratedthroughout. Nearly every facet of the smart home may provide dataassociated with the user (e.g., via the smart device acting as sensorsto provide various data). In some implementations, the attribute dataincludes data corresponding to the home sources 1470. For instance, thehome sources 1470 may include smart appliances, consumables, utilities,and many other smart devices. In a few specific instances, the attributedata may include consumable inventories and consumption rates of variousconsumable goods (e.g., perishables such as milk or bread) tracked,monitored, or otherwise observed by smart refrigerators. In anotherinstance, the attribute data may include utility usage data (e.g.,electricity, water). Analysis of the utility usage data may indicatepatterns or a status of the user, such as the user being on vacation,the user being ill (e.g., increasing house thermostat set temperature tocope with a cold), the user being an energy conscious consumer, and soon.

Referring now to FIG. 15, example diagram 1500 depicts non-limitingexample I/O components 1510 that may provide attribute data, accordingto some example embodiments. In example embodiments, the I/O components1510 include input components 1520, output components 1530,environmental components 1540, motion components 1550, positioncomponents 1560, biometric components 1570, communication components1580, detection components 1590, and a wide gamut of other sensors,gauges, and measurement components not shown in FIG. 15. The I/Ocomponents 1510 or a suitable combination of the I/O components 1510 maybe included in any suitable device or machine such as those included inthe attribute sources 1410 depicted in FIG. 14 to facilitate thefunctionality described herein.

The I/O components 1510 may receive, detect, measure, capture, orotherwise obtain sensor data associated with physical properties,attributes, or characteristics. The I/O components 1510 may provide,produce, transmit, or otherwise communicate the sensor data or otherindications associated with the physical properties, attributes, orcharacteristics (e.g., a sensor included in a device operable tocommunicate the sensor data to the networked system 102). In someimplementations, a combination of devices may be employed to provide thesensor data (e.g., a first device that includes a sensor and iscommunicatively coupled to a second device that communicates sensor datareceived from the first device to the networked system 102). As aresult, the sensor data provided by the I/O components 1510 may beaccessible to all, or some, of the modules described above on areal-time or near real-time basis. The I/O components 1510 are groupedaccording to functionality merely for simplifying the followingdiscussion and the grouping is in no way limiting.

The input components 1520 include alphanumeric input components (e.g., akeyboard, a touch screen configured to receive alphanumeric input, aphoto-optical keyboard, or other alphanumeric input components),point-based input components (e.g., a mouse, a touchpad, a trackball, ajoystick, a motion sensor, or other pointing instrument), tactile inputcomponents (e.g., a physical button, a touch screen that provideslocation and force of touches or touch gestures, or other tactile inputcomponents), audio input components (e.g., a microphone), and the like.In some implementations, the input components 1520 receive input fromthe user to facilitate the functionalities described herein. Forinstance, the user may interact with a user interface using the inputcomponents 1520.

The output components 1530 include visual components (e.g., a displaysuch as a plasma display panel (PDP), a light emitting diode (LED)display, a liquid crystal display (LCD), a projector, or a cathode raytube (CRT)), acoustic components (e.g., speakers), haptic components(e.g., a vibratory motor), other signal generators, and so forth. Theoutput components 1530 may present information to the user. For example,the output components 1530 may present a user interface to the user orpresent media files to the user.

The environmental components 1540 include illumination sensors (e.g.,photometer), temperature sensors (e.g., one or more thermometers thatdetect ambient temperature), humidity sensors, pressure sensors (e.g.,barometer), acoustic sensors (e.g., one or more microphones that detectbackground noise), proximity sensors (e.g., an infrared sensor thatdetects nearby objects), gas sensors (e.g., machine olfaction detectionsensors, gas detection sensors to detect concentrations of hazardousgases for safety or to measure pollutants in the atmosphere), and so on.The environmental components 1540 may measure various physicalparameters to provide an indication or signal corresponding to thephysical environment surrounding the environmental components 1540.

The motion components 1550 include acceleration sensors (e.g.,accelerometer), gravitation sensors, rotation sensors (e.g., gyroscope),and so forth. The motion components 1550 may provide motion data such asvelocity, acceleration, or other force measurements along an x, y, and zaxes. In some implementations, the motion data is provided at a regularupdate rate or sampling rate (e.g., 10 updates per second) that may beconfigurable.

The position components 1560 include location sensors (e.g., a GlobalPosition System (GPS) receiver component), altitude sensors (e.g.,altimeters or barometers that detect air pressure from which altitudemay be derived), orientation sensors (e.g., magnetometers that providemagnetic field strength along the x, y, and z axes), and the like. In anexample embodiment, the position components 1560 may provide positiondata such as latitude, longitude, altitude, and a time stamp. Similar tothe motion components 1550, the position components 1560 may provide themotion data at a regular update rate that may be configurable.

The biometric components 1570 include components to detect expressions,measure biosignals, or identify people, among other functions. Forexample, the biometric components 1570 include expression components todetect expressions (also referred to as “kinesics”) such as handgestures (e.g., an optical component to detect a hand gesture or aDoppler component to detect hand motions), vocal expressions (e.g., amicrophone to detect changes in voice pitch that may indicate tension),facial expressions (e.g., a camera to detect expressions ormicro-expressions of a person such as a smile), body gestures, and eyetracking (e.g., detecting the focal point of a person's eyes or patternsin eye movement). The biometric components 1570 may also include, forexample, biosignal components to measure biosignals such as bloodpressure, heart rate, body temperature, perspiration, and brain waves(e.g., as determined by a electroencephalogram). In further examples,the biometric components 1570 include identification components toidentify people such as retinal scanners (e.g., a camera component),vocal detectors (e.g., a microphone to receive audio data for voiceidentification), facial detectors, fingerprint detectors, andelectroencephalogram sensors (e.g., to identify a person via uniquebrain wave patterns).

Communication may be implemented using a wide variety of technologies.The I/O components 1510 may include communication components 1580operable to communicatively couple machines or devices. For example, thecommunication components 1580 may include a network interface componentor other suitable device to interface with a network (e.g., the network104). In further examples, the communication components 1580 may includewired communication components, wireless communication components,cellular communication components, Near Field Communication (NFC)components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi®components, and other communication components to provide communicationvia other modalities. In addition, a variety of information may bederived using the communication components 1580 such as location viaInternet Protocol (IP) geo-location, location via Wi-Fi® signaltriangulation, location via detecting a NFC beacon signal that mayindicate a particular location, and so forth.

The detection components 1590 provide functionality to detect a varietyof identifiers. For example, the detection components 1590 include RadioFrequency Identification (RFID) tag reader components, Near FieldCommunication (NFC) smart tag detection components, optical readercomponents (e.g., an optical sensor to detect one-dimensional bar codessuch as Universal Product Code (UPC) bar codes, multi-dimensional barcodes such as a Quick Response (QR) code, Aztec code, Data Matrix,Dataglyph, MaxiCode, PDF417, Ultra Code, Uniform Commercial Code ReducedSpace Symbology (UCC RSS)-2D bar code, and other optical codes), oracoustic detection components (e.g., microphones to identify taggedaudio signals). In additional, a variety of information may be derivedvia various communication components such as location via InternetProtocol (IP) geo-location, location via Wi-Fi® signal triangulation,location via detecting a NFC beacon signal that may indicate aparticular location, and so forth.

FIG. 16 is a block diagram 1600 of an example data structure for theattribute data associated with a particular user according to exampleembodiments. In embodiments, the attribute data is associated with aplurality of users such as user 1602, 1604, 1606, 1608, 1610, 1612, and1614. In an embodiment, the attribute data is accessed for a particularuser via a lookup using a user identifier. The attribute data includes,for example, profile data 1620, device data 1622, calendar data 1624,list data 1626, list type data 1628, interest data 1630, fitment data1632, garment type data 1634, preference data 1636, measured dimensiondata 1638, fitness goal data 1640, reward data 1642, location data 1644,and other data not shown in FIG. 16. In some embodiments, the attributedata may be structured such that various portions of the attribute dataare associated with other portions of the attribute data viarelationships. For instance, the calendar data 1624 may include acalendar event associated with an event name, an event data, and anevent location for the calendar event.

FIG. 17 is a block diagram 1700 of an example data structure for dataassociated with a device according to some example embodiments. In anexample embodiment, the device data 1622 of FIG. 16 may include a deviceidentifier, a device name, device resources data (e.g., files stores onthe devices such as browser cookies, media files), I/O component data,and so forth. In example embodiments, the device identifier comprises,for example, an Internet Protocol (IP) address, a Media Access Control(MAC) address, other unique identifiers, an International Mobile StationEquipment Identity (IMEI), or a Mobile Equipment Identifier (MEID). Inan embodiment, the I/O component data includes standard deviceparameters 1702, position data 1704, location data 1706, motion data1708, environmental data 1710, biometric data 1712, among other data.FIG. 17 merely depicts example attribute data that may correspond to aparticular device, and a variety of other data not shown in FIG. 17 maybe included in the device data. In various embodiments, the standarddevice parameters 1702 include parameters that are standard acrossmultiple devices included in the IoT. In some embodiments, standardizedparameters and protocols facilitate access and utilization of theattribute data corresponding to such devices. For example, the attributedata available on an unknown device may be accessed and utilized withoutthe need to discover or otherwise determine which parameters areavailable and which units of measure are associated with the parameters.Many other schemes may be employed to discover or otherwise determineavailable parameters accessible on a particular device.

Modules, Components, and Logic

Certain embodiments are described herein as including logic or a numberof components, modules, or mechanisms. Modules may constitute eithersoftware modules (e.g., code embodied on a machine-readable medium or ina transmission signal) or hardware modules. A “hardware module” is atangible unit capable of performing certain operations and may beconfigured or arranged in a certain physical manner. In various exampleembodiments, one or more computer systems (e.g., a standalone computersystem, a client computer system, or a server computer system) or one ormore hardware modules of a computer system (e.g., a processor or a groupof processors) may be configured by software (e.g., an application orapplication portion) as a hardware module that operates to performcertain operations as described herein.

In some embodiments, a hardware module may be implemented mechanically,electronically, or any suitable combination thereof. For example, ahardware module may include dedicated circuitry or logic that ispermanently configured to perform certain operations. For example, ahardware module may be a special-purpose processor, such as aField-Programmable Gate Array (FPGA) or an Application SpecificIntegrated Circuit (ASIC). A hardware module may also includeprogrammable logic or circuitry that is temporarily configured bysoftware to perform certain operations. For example, a hardware modulemay include software encompassed within a general-purpose processor orother programmable processor. It will be appreciated that the decisionto implement a hardware module mechanically, in dedicated andpermanently configured circuitry, or in temporarily configured circuitry(e.g., configured by software) may be driven by cost and timeconsiderations.

Accordingly, the phrase “hardware module” should be understood toencompass a tangible entity, be that an entity that is physicallyconstructed, permanently configured (e.g., hardwired), or temporarilyconfigured (e.g., programmed) to operate in a certain manner or toperform certain operations described herein. As used herein,“hardware-implemented module” refers to a hardware module. Consideringembodiments in which hardware modules are temporarily configured (e.g.,programmed), each of the hardware modules need not be configured orinstantiated at any one instance in time. For example, where a hardwaremodule comprises a general-purpose processor configured by software tobecome a special-purpose processor, the general-purpose processor may beconfigured as respectively different special-purpose processors (e.g.,comprising different hardware modules) at different times. Software mayaccordingly configure a particular processor or processors, for example,to constitute a particular hardware module at one instance of time andto constitute a different hardware module at a different instance oftime.

Hardware modules can provide information to, and receive informationfrom, other hardware modules. Accordingly, the described hardwaremodules may be regarded as being communicatively coupled. Where multiplehardware modules exist contemporaneously, communications may be achievedthrough signal transmission (e.g., over appropriate circuits and buses)between or among two or more of the hardware modules. In embodiments inwhich multiple hardware modules are configured or instantiated atdifferent times, communications between such hardware modules may beachieved, for example, through the storage and retrieval of informationin memory structures to which the multiple hardware modules have access.For example, one hardware module may perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware module may then, at a latertime, access the memory device to retrieve and process the storedoutput. Hardware modules may also initiate communications with input oroutput devices, and can operate on a resource (e.g., a collection ofinformation).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions describedherein. As used herein, “processor-implemented module” refers to ahardware module implemented using one or more processors.

Similarly, the methods described herein may be at least partiallyprocessor-implemented, with a particular processor or processors beingan example of hardware. For example, at least some of the operations ofa method may be performed by one or more processors orprocessor-implemented modules. Moreover, the one or more processors mayalso operate to support performance of the relevant operations in a“cloud computing” environment or as a “software as a service” (SaaS).For example, at least some of the operations may be performed by a groupof computers (as examples of machines including processors), with theseoperations being accessible via a network (e.g., the Internet) and viaone or more appropriate interfaces (e.g., an Application ProgramInterface (API)).

The performance of certain of the operations may be distributed amongthe processors, not only residing within a single machine, but deployedacross a number of machines. In some example embodiments, the processorsor processor-implemented modules may be located in a single geographiclocation (e.g., within a home environment, an office environment, or aserver farm). In other example embodiments, the processors orprocessor-implemented modules may be distributed across a number ofgeographic locations.

Software Architecture

FIG. 18 is a block diagram 1800 illustrating an architecture of software1802, which may be installed on any one or more of the devices describedabove. FIG. 18 is merely a non-limiting example of a softwarearchitecture, and it will be appreciated that many other architecturesmay be implemented to facilitate the functionality described herein. Thesoftware 1802 may be implemented by hardware such as machine 1900 ofFIG. 19 that includes processors 1910, memory 1930, and I/O components1950. In this example architecture, the software 1802 may beconceptualized as a stack of layers where each layer may provide aparticular functionality. For example, the software 1802 includes layerssuch as an operating system 1804, libraries 1806, frameworks 1808, andapplications 1810. Operationally, the applications 1810 invokeapplication programming interface (API) calls 1812 through the softwarestack and receive messages 1814 in response to the API calls 1812,according to some implementations.

In various implementations, the operating system 1804 manages hardwareresources and provides common services. The operating system 1804includes, for example, a kernel 1820, services 1822, and drivers 1824.The kernel 1820 acts as an abstraction layer between the hardware andthe other software layers in some implementations. For example, thekernel 1820 provides memory management, processor management (e.g.,scheduling), component management, networking, security settings, amongother functionality. The services 1822 may provide other common servicesfor the other software layers. The drivers 1824 may be responsible forcontrolling or interfacing with the underlying hardware. For instance,the drivers 1824 may include display drivers, camera drivers, Bluetooth®drivers, flash memory drivers, serial communication drivers (e.g.,Universal Serial Bus (USB) drivers), Wi-Fi® drivers, audio drivers,power management drivers, and so forth.

In some implementations, the libraries 1806 provide a low-level commoninfrastructure that may be utilized by the applications 1810. Thelibraries 1806 may include system 1830 libraries (e.g., C standardlibrary) that may provide functions such as memory allocation functions,string manipulation functions, mathematic functions, and the like. Inaddition, the libraries 1806 may include API libraries 1832 such asmedia libraries (e.g., libraries to support presentation andmanipulation of various media formats such as Moving Picture ExpertsGroup-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving PictureExperts Group Layer-3 (MP3), Advanced Audio Coding (AAC), AdaptiveMulti-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG orJPG), Portable Network Graphics (PNG)), graphics libraries (e.g., anOpenGL framework used to render in two dimensions (2D) and threedimensions (3D) in a graphic content on a display), database libraries(e.g., SQLite to provide various relational database functions), weblibraries (e.g., WebKit to provide web browsing functionality), and thelike. The libraries 1806 may also include a wide variety of otherlibraries 1834 to provide many other APIs to the applications 1810.

The frameworks 1808 provide a high-level common infrastructure that maybe utilized by the applications 1810, according to some implementations.For example, the frameworks 1808 provide various graphic user interface(GUI) functions, high-level resource management, high-level locationservices, and so forth. The frameworks 1808 may provide a broad spectrumof other APIs that may be utilized by the applications 1810, some ofwhich may be specific to a particular operating system or platform.

In an example embodiment, the applications 1810 include a homeapplication 1850, a contacts application 1852, a browser application1854, a book reader application 1856, a location application 1858, amedia application 1860, a messaging application 1862, a game application1864, and a broad assortment of other applications such as third partyapplication 1866. According to some embodiments, the applications 1810are programs that execute functions defined in the programs. Variousprogramming languages may be employed to create one or more of theapplications 1810, structured in a variety of manners, such asobject-orientated programming languages (e.g., Objective-C, Java, orC++) or procedural programming languages (e.g., C or assembly language).In a specific example, the third party application 1866 (e.g., anapplication developed using the Android™ or iOS™ software developmentkit (SDK) by an entity other than the vendor of the particular platform)may be mobile software running on a mobile operating system such asiOS™, Android™, Windows® Phone, or other mobile operating systems. Inthis example, the third party application 1866 may invoke the API calls1812 provided by the mobile operating system 1804 to facilitatefunctionality described herein.

Example Machine Architecture and Machine-Readable Medium

FIG. 19 is a block diagram illustrating components of a machine 1900,according to some example embodiments, able to read instructions from amachine-readable medium (e.g., a machine-readable storage medium) andperform any one or more of the methodologies discussed herein.Specifically, FIG. 19 shows a diagrammatic representation of the machine1900 in the example form of a computer system, within which instructions1916 (e.g., software, a program, an application, an applet, an app, orother executable code) for causing the machine 1900 to perform any oneor more of the methodologies discussed herein may be executed. Inalternative embodiments, the machine 1900 operates as a standalonedevice or may be coupled (e.g., networked) to other machines. In anetworked deployment, the machine 1900 may operate in the capacity of aserver machine or a client machine in a server-client networkenvironment, or as a peer machine in a peer-to-peer (or distributed)network environment. The machine 1900 may comprise, but not be limitedto, a server computer, a client computer, a personal computer (PC), atablet computer, a laptop computer, a netbook, a set-top box (STB), apersonal digital assistant (PDA), an entertainment media system, acellular telephone, a smart phone, a mobile device, a wearable device(e.g., a smart watch), a smart home device (e.g., a smart appliance),other smart devices, a web appliance, a network router, a networkswitch, a network bridge, or any machine capable of executing theinstructions 1916, sequentially or otherwise, that specify actions to betaken by machine 1900. Further, while only a single machine 1900 isillustrated, the term “machine” shall also be taken to include acollection of machines 1900 that individually or jointly execute theinstructions 1916 to perform any one or more of the methodologiesdiscussed herein.

The machine 1900 may include processors 1910, memory 1930, and I/Ocomponents 1950, which may be configured to communicate with each othervia a bus 1902. In an example embodiment, the processors 1910 (e.g., aCentral Processing Unit (CPU), a Reduced Instruction Set Computing(RISC) processor, a Complex Instruction Set Computing (CISC) processor,a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), anApplication Specific Integrated Circuit (ASIC), a Radio-FrequencyIntegrated Circuit (RFIC), another processor, or any suitablecombination thereof) may include, for example, processor 1912 andprocessor 1914 that may execute instructions 1916. The term “processor”is intended to include multi-core processors that may comprise two ormore independent processors (also referred to as “cores”) that mayexecute instructions contemporaneously. Although FIG. 19 shows multipleprocessors, the machine 1900 may include a single processor with asingle core, a single processor with multiple cores (e.g., a multi-coreprocess), multiple processors with a single core, multiple processorswith multiples cores, or any combination thereof.

The memory 1930 may include a main memory 1932, a static memory 1934,and a storage unit 1936 accessible to the processors 1910 via the bus1902. The storage unit 1936 may include a machine-readable medium 1938on which is stored the instructions 1916 embodying any one or more ofthe methodologies or functions described herein. The instructions 1916may also reside, completely or at least partially, within the mainmemory 1932, within the static memory 1934, within at least one of theprocessors 1910 (e.g., within the processor's cache memory), or anysuitable combination thereof, during execution thereof by the machine1900. Accordingly, in various implementations, the main memory 1932,static memory 1934, and the processors 1910 are considered asmachine-readable media 1938.

As used herein, the term “memory” refers to a machine-readable medium1938 able to store data temporarily or permanently and may be taken toinclude, but not be limited to, random-access memory (RAM), read-onlymemory (ROM), buffer memory, flash memory, and cache memory. While themachine-readable medium 1938 is shown in an example embodiment to be asingle medium, the term “machine-readable medium” should be taken toinclude a single medium or multiple media (e.g., a centralized ordistributed database, or associated caches and servers) able to storeinstructions 1916. The term “machine-readable medium” shall also betaken to include any medium, or combination of multiple media, that iscapable of storing instructions (e.g., instructions 1916) for executionby a machine (e.g., machine 1900), such that the instructions, whenexecuted by one or more processors of the machine 1900 (e.g., processors1910), cause the machine 1900 to perform any one or more of themethodologies described herein. Accordingly, a “machine-readable medium”refers to a single storage apparatus or device, as well as “cloud-based”storage systems or storage networks that include multiple storageapparatus or devices. The term “machine-readable medium” shallaccordingly be taken to include, but not be limited to, one or more datarepositories in the form of a solid-state memory (e.g., flash memory),an optical medium, a magnetic medium, other non-volatile memory (e.g.,Erasable Programmable Read-Only Memory (EPROM)), or any suitablecombination thereof. The term “machine-readable medium” specificallyexcludes non-statutory signals per se.

The I/O components 1950 include a wide variety of components to receiveinput, provide output, produce output, transmit information, exchangeinformation, capture measurements, and so on. In general, it will beappreciated that the I/O components 1950 may include many othercomponents that are not shown in FIG. 19. The I/O components 1950 aregrouped according to functionality merely for simplifying the followingdiscussion and the grouping is in no way limiting. In various exampleembodiments, the I/O components 1950 include output components 1952 andinput components 1954. The output components 1952 include visualcomponents (e.g., a display such as a plasma display panel (PDP), alight emitting diode (LED) display, a liquid crystal display (LCD), aprojector, or a cathode ray tube (CRT)), acoustic components (e.g.,speakers), haptic components (e.g., a vibratory motor), other signalgenerators, and so forth. The input components 1954 include alphanumericinput components (e.g., a keyboard, a touch screen configured to receivealphanumeric input, a photo-optical keyboard, or other alphanumericinput components), point based input components (e.g., a mouse, atouchpad, a trackball, a joystick, a motion sensor, or other pointinginstrument), tactile input components (e.g., a physical button, a touchscreen that provides location and force of touches or touch gestures, orother tactile input components), audio input components (e.g., amicrophone), and the like.

In some further example embodiments, the I/O components 1950 includebiometric components 1956, motion components 1958, environmentalcomponents 1960, or position components 1962 among a wide array of othercomponents. For example, the biometric components 1956 includecomponents to detect expressions (e.g., hand expressions, facialexpressions, vocal expressions, body gestures, or eye tracking), measurebiosignals (e.g., blood pressure, heart rate, body temperature,perspiration, or brain waves), identify a person (e.g., voiceidentification, retinal identification, facial identification,fingerprint identification, or electroencephalogram basedidentification), and the like. The motion components 1958 includeacceleration sensor components (e.g., accelerometer), gravitation sensorcomponents, rotation sensor components (e.g., gyroscope), and so forth.The environmental components 1960 include, for example, illuminationsensor components (e.g., photometer), temperature sensor components(e.g., one or more thermometer that detect ambient temperature),humidity sensor components, pressure sensor components (e.g.,barometer), acoustic sensor components (e.g., one or more microphonesthat detect background noise), proximity sensor components (e.g.,infrared sensors that detect nearby objects), gas sensors (e.g., machineolfaction detection sensors, gas detection sensors to detectionconcentrations of hazardous gases for safety or to measure pollutants inthe atmosphere), or other components that may provide indications,measurements, or signals corresponding to a surrounding physicalenvironment. The position components 1962 include location sensorcomponents (e.g., a Global Position System (GPS) receiver component),altitude sensor components (e.g., altimeters or barometers that detectair pressure from which altitude may be derived), orientation sensorcomponents (e.g., magnetometers), and the like.

Communication may be implemented using a wide variety of technologies.The I/O components 1950 may include communication components 1964operable to couple the machine 1900 to a network 1980 or devices 1970via coupling 1982 and coupling 1972, respectively. For example, thecommunication components 1964 include a network interface component oranother suitable device to interface with the network 1980. In furtherexamples, communication components 1964 include wired communicationcomponents, wireless communication components, cellular communicationcomponents, Near Field Communication (NFC) components, Bluetooth®components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and othercommunication components to provide communication via other modalities.The devices 1970 may be another machine or any of a wide variety ofperipheral devices (e.g., a peripheral device coupled via a UniversalSerial Bus (USB)).

Moreover, in some implementations, the communication components 1964detect identifiers or include components operable to detect identifiers.For example, the communication components 1964 include Radio FrequencyIdentification (RFID) tag reader components, NFC smart tag detectioncomponents, optical reader components (e.g., an optical sensor to detecta one-dimensional bar codes such as Universal Product Code (UPC) barcode, multi-dimensional bar codes such as Quick Response (QR) code,Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code,Uniform Commercial Code Reduced Space Symbology (UCC RSS)-2D bar code,and other optical codes), acoustic detection components (e.g.,microphones to identify tagged audio signals), or any suitablecombination thereof. In addition, a variety of information can bederived via the communication components 1964, such as, location viaInternet Protocol (IP) geo-location, location via Wi-Fi® signaltriangulation, location via detecting a NFC beacon signal that mayindicate a particular location, and so forth.

Transmission Medium

In various example embodiments, one or more portions of the network 1980may be an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), the Internet, a portion of the Internet, a portion of the PublicSwitched Telephone Network (PSTN), a plain old telephone service (POTS)network, a cellular telephone network, a wireless network, a Wi-Fi®network, another type of network, or a combination of two or more suchnetworks. For example, the network 1980 or a portion of the network 1980may include a wireless or cellular network and the coupling 1982 may bea Code Division Multiple Access (CDMA) connection, a Global System forMobile communications (GSM) connection, or other type of cellular orwireless coupling. In this example, the coupling 1982 may implement anyof a variety of types of data transfer technology, such as SingleCarrier Radio Transmission Technology (1×RTT), Evolution-Data Optimized(EVDO) technology, General Packet Radio Service (GPRS) technology,Enhanced Data rates for GSM Evolution (EDGE) technology, thirdGeneration Partnership Project (3GPP) including 3G, fourth generationwireless (4G) networks, Universal Mobile Telecommunications System(UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability forMicrowave Access (WiMAX), Long Term Evolution (LTE) standard, othersdefined by various standard setting organizations, other long rangeprotocols, or other data transfer technology.

In example embodiments, the instructions 1916 are transmitted orreceived over the network 1980 using a transmission medium via a networkinterface device (e.g., a network interface component included in thecommunication components 1964) and utilizing any one of a number ofwell-known transfer protocols (e.g., Hypertext Transfer Protocol(HTTP)). Similarly, in other example embodiments, the instructions 1916are transmitted or received using a transmission medium via the coupling1972 (e.g., a peer-to-peer coupling) to devices 1970. The term“transmission medium” shall be taken to include any intangible mediumthat is capable of storing, encoding, or carrying instructions 1916 forexecution by the machine 1900, and includes digital or analogcommunications signals or other intangible medium to facilitatecommunication of such software.

Furthermore, the machine-readable medium 1938 is non-transitory (inother words, not having any transitory signals) in that it does notembody a propagating signal. However, labeling the machine-readablemedium 1938 as “non-transitory” should not be construed to mean that themedium is incapable of movement; the medium should be considered asbeing transportable from one physical location to another. Additionally,since the machine-readable medium 1938 is tangible, the medium may beconsidered to be a machine-readable device.

Language

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be implemented as a combined structure or component. Similarly,structures and functionality presented as a single component may beimplemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

Although an overview of the inventive subject matter has been describedwith reference to specific example embodiments, various modificationsand changes may be made to these embodiments without departing from thebroader scope of embodiments of the present disclosure. Such embodimentsof the inventive subject matter may be referred to herein, individuallyor collectively, by the term “invention” merely for convenience andwithout intending to voluntarily limit the scope of this application toany single disclosure or inventive concept if more than one is, in fact,disclosed.

The embodiments illustrated herein are described in sufficient detail toenable those skilled in the art to practice the teachings disclosed.Other embodiments may be used and derived therefrom, such thatstructural and logical substitutions and changes may be made withoutdeparting from the scope of this disclosure. The Detailed Description,therefore, is not to be taken in a limiting sense, and the scope ofvarious embodiments is defined only by the appended claims, along withthe full range of equivalents to which such claims are entitled.

As used herein, the term “or” may be construed in either an inclusive orexclusive sense. Moreover, plural instances may be provided forresources, operations, or structures described herein as a singleinstance. Additionally, boundaries between various resources,operations, modules, engines, and data stores are somewhat arbitrary,and particular operations are illustrated in a context of specificillustrative configurations. Other allocations of functionality areenvisioned and may fall within a scope of various embodiments of thepresent disclosure. In general, structures and functionality presentedas separate resources in the example configurations may be implementedas a combined structure or resource. Similarly, structures andfunctionality presented as a single resource may be implemented asseparate resources. These and other variations, modifications,additions, and improvements fall within a scope of embodiments of thepresent disclosure as represented by the appended claims. Thespecification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense.

What is claimed is:
 1. A method comprising: detecting a first activitybeing performed at a first device associated with a user, the firstactivity including consumption of first content received from a contentprovider over a network; identifying a second device associated with theuser by detecting a short range communications connection between thefirst device and the second device, the second device being currentlycapable of performing a second activity that is unavailable forperformance by the first device; determining that the second activity isto be performed for the user during the first activity by inferring auser preference from attribute data associated with the user; subsequentto identifying the second device, generating instructions that enablethe second device to perform the second activity based on data collectedvia a sensor of the second device describing a physical environment inwhich the second device is disposed; and causing the second device toperform the second activity by transmitting the instructions to thesecond device.
 2. The method of claim 1, wherein the short rangecommunications connection comprises a Bluetooth connection or a NearField Communication (NFC) connection.
 3. The method of claim 1, wherein:the first device comprises one of a smart phone, a tablet computer, or alaptop computer configured to display the first content in twodimensions; and the second device comprises a wearable device configuredto display second content in three dimensions and to enable userinteraction with the second content displayed in three dimensions,wherein the second activity comprises displaying the second content. 4.The method of claim 1, wherein identifying the second device is furtherdetermined based on a determination that the sensor at the second deviceis currently collecting the data describing the physical environment inwhich the second device is disposed.
 5. The method of claim 1, whereinthe second activity comprises a notification associated with the firstactivity, the notification including haptic output.
 6. A devicecomprising: one or more hardware processors; and a memory storinginstructions that, when executed by at least one of the one or morehardware processors, cause the device to perform operations comprising:detecting a first activity being performed at the device, the firstactivity including consumption of first content received from a contentprovider over a network; identifying a different device associated withthe device by detecting a short range communications connection betweenthe device and the different device, the different device beingcurrently capable of performing a second activity, that is unavailablefor performance by the device; determining that the second activity isto be performed at the different device during the first activity byinferring a user preference for a user of the device from attribute dataassociated with the user; subsequent to identifying the differentdevice, generating instructions that enable the different device toperform the second activity based on data collected via a sensor of thedifferent device describing a physical environment in which thedifferent device is disposed; and causing the different device toperform the second activity by transmitting the instructions to thedifferent device.
 7. The device of claim 6, wherein the short rangecommunications connection comprises a Bluetooth connection or a NearField Communication (NFC) connection.
 8. The device of claim 6, wherein:the device comprises one of a smart phone, a tablet computer, or alaptop computer configured to display the first content in twodimensions; and the different device comprises a wearable deviceconfigured to display second content in three dimensions and to enableuser interaction with the second content displayed in three dimensions,wherein the second activity comprises displaying the second content. 9.The device of claim 6, wherein identifying the different device isfurther performed based on a determination that the sensor of thedifferent device is currently collecting the data describing thephysical environment in which the different device is disposed.
 10. Thedevice of claim 6, wherein the second activity comprises a notificationassociated with the first activity, the notification including hapticoutput.
 11. A computer-readable storage medium storing instructionsthat, when executed by at least one hardware processor of a firstdevice, cause the first device to perform operations comprising:detecting a first activity being performed at the first device, thefirst activity including consumption of first content received from acontent provider over a network; identifying a second device associatedwith the first device by detecting a short range communicationsconnection between the first device and the second device, the seconddevice being currently capable of performing a second activity that isunavailable for performance by the first device; determining that thesecond activity is to be performed during the first activity byinferring a user preference for a user of the first device fromattribute data associated with the uer; subsequent to identifying thesecond device, identifying instructions for the second device to performthe second activity based on data collected via a sensor of the seconddevice describing a physical environment in which the second device isdisposed; and causing the second device to perform the second activityby causing the instructions to be transmitted to the second device. 12.The computer-readable storage medium of claim 11, wherein the shortrange communications connection is a Bluetooth connection or a NearField Communication (NFC) connection.
 13. The computer-readable storagemedium of claim 11, wherein: the first device comprises one of a smartphone, a tablet computer, or a laptop computer configured to display thefirst content in two dimensions; and the second device comprises awearable device configured to display second content in three dimensionsand to enable user interaction with the second content displayed inthree dimensions, wherein the second activity comprises displaying thesecond content.
 14. The computer-readable storage medium of claim 11,wherein identifying the second device is further performed based on adetermination that the sensor is currently collecting the datadescribing the physical environment in which the second device isdisposed.
 15. The computer-readable storage medium of claim 11, whereinthe second activity comprises a notification associated with the firstactivity, the notification including haptic output.
 16. Thecomputer-readable storage medium of claim 11, wherein the data collectedvia the sensor of the second device describes a temperature of thephysical environment.
 17. The computer-readable storage medium of claim11, wherein the data collected via the sensor of the second devicedescribes motion of the second device in the physical environment. 18.The computer-readable storage medium of claim 11, wherein the datacollected via the sensor of the second device describes position datafor the second device relative to the physical environment.
 19. Thecomputer-readable storage medium of claim 11, wherein the data collectedvia the sensor of the second device describes biometric data for theuser.
 20. The computer-readable storage medium of claim 11, theoperations further comprising generating an active metric using thesensor data and comparing the active metric to a threshold value,wherein generating the instructions is performed responsive todetermining that the active metric satisfies the threshold value.